In the modern-day production of motion pictures, the use of special effects is becoming much more common. Some types of special effects involve the merging of a "live" shot of an actor or scene with a computer-generated image. One example of this type of effect employs a virtual set, or virtual studio, in which an actor is filmed, or taped, while performing in front of a so-called "blue screen", which provides a background of a uniform color that can be easily filtered out during subsequent production processes. The recorded motions of the actor can then be superimposed on a computer-generated scene, to provide the final result.
In another type of special effect, a computer-generated element is merged into the image that has been recorded by the camera. For example, a recorded scene may contain a background building that is only a few stories tall. To create the impression that the building is much taller, a computer-generated image of several more stories can be added to the original image of the building. As another example, an animated character can be incorporated into a scene.
In the production of these types of special effects, it is necessary to know the position of the camera during the recording of the live action. For example, if the camera moves from a position directly in front of an actor to an oblique angle, the perspective from which the computer-generated scene is viewed in the final image must change in a commensurate manner, to provide a realistic impression. In other words, the rendering of the computer-generated image at every instant in time must be consistent with the location and orientation of the camera during the filming of live action at that same instant.
In the past, the recording of the camera's position has been carried out in several different ways. In one approach, a special virtual set is constructed whose walls contain gradations that can be optically sensed and processed to provide position information. For example, a unique pattern of gradations can be marked on the blue screen which forms a wall of the virtual set. By using pattern recognition techniques, the location and orientation of the camera can be determined at all times. One example of this type of virtual set is the CYBERSET O sold by Orad Hi-Tech Systems, Ltd. This approach to camera position sensing is restricted to use within the specially constructed studio, and therefore cannot be employed when filming scenes outdoors, or in any interior environment other than a virtual set.
To avoid the limitations of having to use a virtual set, another optically-oriented approach employs image processing techniques to recognize objects in a scene. By interpreting the relative sizes and positions of objects in the recorded scene, the location and orientation of the camera can be determined. This technique is difficult to implement at a practical level, however, and therefore is not regularly employed.
In another known approach, the camera is mounted upon a rigid, encoder-equipped platform. The encoders on the platform enable the position of the camera to be precisely monitored at all times. The advantage offered by this approach is that it can be used outdoors, for example by placing the platform in the area where a scene is to be filmed. However, since the camera is required to be mounted on a rigid structure at all times, this approach is not suitable for hand-held cameras and the like. As a result, restrictions are placed on the range of movement of the camera during the filming of the scene.
It is an objective of the present invention, therefore, to provide a system for accurately sensing the position of a camera which can be readily used in conjunction with hand-held cameras, and yet which does not require a special environment, such as a blue-screen studio, and therefore can be used to record almost any type of action and/or scene.